﻿IEEE 10th International Symposium on Wearable Computers Montreaux, Switzerland October, 2006 Human Activity Recognition: Accuracy across Common Locations for Wearable Sensors Daniel Olgu´n Olgu´n, Alex (Sandy) Pentland MIT Media Laboratory, Human Dynamics Group 20 Ames St Room E15-383 Cambridge, MA 02139 {dolguin, sandy}@media mit edu on different parts of the body: right wrist (A1), left hip (A2), Abstract and chest (A3) We selected these locations on the basis of lengthy discussions with three potential user groups (of- In recent years much work has been done on human ce workers, hospital patients, and army soldiers) concern- activity recognition using wearable sensors As we begin ing their general acceptance of wearable sensors We found to deploy hundreds and even thousands of wearable sen- that there was broad consensus about the acceptability of the sors on regular workers, hospital patients, and army sol- chest and hip locations, and so we are developing a wear- diers, the question shifts more toward a balance between able electronic badge that will be worn on the chest This what information can be gained and their broad immedi- badge will be able to share information with a Bluetooth- ate user acceptance In this paper we compare the activ- enabled cellular phone that could be worn on the hip and ity classiﬁcation accuracy of four different conﬁgurations might have a second accelerometer of accelerometer placement on the human body using hid- den Markov models (HMMs) We ﬁnd the classiﬁcation ac- curacy of a single accelerometer placed in three different 2 Related Work parts of the body and evaluate whether there is a signiﬁ- cant improvement in recognition accuracy by adding multi- Previous work in human activity recognition using ac- ple accelerometers or not We also ﬁnd the number of hid- celerometers has shown that it is possible to classify several den states that best models each activity by achieving the postures and activities in real time In , the authors de- lowest test error using K-fold cross-validation veloped a two-layer model that combined a Gaussian mix- ture model with rst-order Markov models to classify a range of activities including: sitting, walking, biking, and 1 Introduction riding the subway A single 3-axis accelerometer placed on the torso was used In , the authors combined data from three accelerometers and two microphones placed on Being able to automatically recognize human motion different body locations to classify activities performed in patterns using unobtrusive wearable sensors can be useful in a wood shop with 84 4% accuracy They modeled most of monitoring the elderly in their homes and keep track of their the activities using single Gaussian hidden Markov models daily activities and behavioral changes This could lead to The number of hidden states to model each activity was se- a better understanding of numerous medical conditions and lected through visual inspection In , several algorithms treatments Other applications of human activity recogni- to classify twenty different physical activities from data ac- tion range from context aware computing to physical train- quired using ve 2-axis accelerometers were evaluated with ing, physical rehabilitation, and military applications such an overall recognition rate of 84% as intelligent outt design for soldiers In , the authors used decision trees to classify six In this paper we study different congurations of ac- different activities with a single accelerometer, placed on celerometer placement to classify human activities that are six different body positions commonly used for wearing frequent in at least one of these application areas Eight dif- electronic devices, with accuracies ranging from 16 7% to ferent activities were modeled using HMMs and continuous 92 8% depending on the position of the accelerometer and Gaussian observation vectors Three wireless accelerome- the features used ters (MITes: MIT Environmental Sensors) were placed3 Data Collection and Processing The nal classication was obtained as ˆl G(O) = arg maxL(c) (1) Three different subjects were asked to perform the fol- c lowing sequence of activities: 1) Sit down, 2) Run, 3) Squat, ˆ 4) Walk, 5) Stand, 6) Crawl, 7) Lay down (on the chest), The classierGtakes values in the class setG= and 8) Hand movements (while standing) The data col- {1,2, C} lection process was repeated three times for each subject This process was repeatedK =9times using k-fold 90 seconds of data were collected for each activity and la- cross-validation The average cross-validation classication beled according to the start and stop times of each activity accuracy per class is compared for the four possible cong- performed The data were divided into nine datasets, each urations of accelerometer placement shown in table 1 of them containing 80 observation sequences of one-second duration An HMM was trained for each activity using eight Conguration datasets (640 observations) and tested on the ninth dataset C1C2C3C4 (80 observations) This process was repeated nine times, Right wristYESYESYES each time using different training and test sets to obtain the Left hipYESYESYES K-fold cross-validation classication accuracy ChestYESYESYES We resampled each dataset atf0=50Hz mean s The and variance of the acceleration in thex, y, andz axes from Table 1 Accelerometer conﬁgurations each of the three accelerometers were calculated over 200- millisecond time slices 5 Results 4 Activity Classiﬁcation Using HMMs Figure 1 shows a comparison of classication accuracy when a single accelerometer was used for activity classi- Given a training set of labeled observation sequences cation We are able to discern activities such as walking (features extracted from the acceleration readings in thex, (65 68%) and performing hand movements (56 30%) using y, andz axes from three accelerometers placed on different only accelerometer A1 (right wrist) Accelerometer A2 (left parts of the body), corresponding to each of the activities hip) played the most important role when classifying ac- that we want to classify, we rst want to estimate the model tivities such as sitting(66 05%), running (97 78%), crawl- parameters =(A,B,), whereA={aij} is the the ing (69 26%), and lying down (87 04%) Accelerometer A3 state transition probability distribution,B={bj(k)} is the (chest) was best for classifying activities such as squatting observation symbol probability distribution in statej, and (75 8%) and standing (77 78%) ={} is initial state distribution for each activity i the Given a new set of observations we would like to classify Classification Accuracy Comparison 100 each sequence according to the model that gives the maxi- A1 A2 90 A3 80 mum likelihood for that particular sequence 70 We modeled each observation sequence as a 5-state left- 60 50 to-right HMM with continuous Gaussian observation vec- 40 30 tors and two hidden states Each observation vector was Cross-Validation Accuracy 20 10 formed by combining the mean and variance in thex,y, 0 12345678 andzaxes from each accelerometer These features were Running Squatting Walking Sitting CrawlingHand mvt StandingLying down previously used in An HMM was trained for each class(1,2, , C), whereindicates the learned HMM cFigure 1 K-fold cross validation accuracy model for classc, andC =8is the total number of classes, comparison using a single accelerometer using the labeled data from eight datasets as training dataset T ninth data set was used as the validation datasetV k Thek Figure 2 shows the average classication accuracy per The HMM toolbox for Matlab developed by was activity when combinations of two and three accelerometers used to train and test the different models The log- were placed in the four different congurations described in likelihood of each model was calculated for each obser- table 1 Table 2 shows a comparison between the classi- vation sequence in the ninth dataset Each observation se- cation accuracy obtained when a single accelerometer (A1 quenceOl={OlOl Ol} (with slices) in the 12T T =5time to A3) was used, and the classication accuracy obtained validation datasetV={Ol}Lwas according kl=1 classied when multiple accelerometers were used (C1 to C4) to the model that gave the maximum likelihood Classification Accuracy Comparison 100 ClassQ =2QminVa rQopt C1 90 C2 C3 197 78%93 33%-4 45%2 80 C4 70 298 89%98 89%0%2 60 50 382 72%82 72%0%2 40 30 484 20%88 64%+4 44%4 Cross-Validation Accuracy 20 595 56%96 67%+1 11%4 10 0 12345678 691 98%86 29%-5 69%2 Running Squatting Walking Sitting CrawlingHand mvt StandingLying down 787 16%87 26%+0 1%4 898 77%99 01%+0 24%5 Figure 2 K-fold cross validation accuracy comparison for the four different conﬁgura- Table 3 Classiﬁcation accuracy variation and tions of accelerometer placement optimal number of hidden states ClassA1A2A3C1C2C3C4 6 Conclusions 158 40%66 05%63 21%88 52%94 44%95 56%97 78% 280 86%97 78%96 54%98 89%95 43%100 0%98 89% We found that the best global classication performance 323 46%42 72%75 80%72 10%67 90%92 72%82 72% was achieved when using conguration C4 (92 13%), al- 465 68%41 73%54 44%84 32%77 28%61 11%84 20% though it might be possible to obtain similar results using 569 75%68 77%77 78%90 00%94 44%97 65%95 56% only two accelerometers We showed that the global clas- 645 93%69 26%57 65%84 20%69 26%73 58%91 98% sication accuracy that can be achieved using a single ac- 743 21%87 04%41 98%81 11%76 42%86 05%87 16% celerometer is around 60%, and determined the activities 856 30%53 51%32 22%98 77%79 85%55 28%98 77% that are best classied with each accelerometer placed on three different parts of the body Modeling each activity Global55 45%65 86%62 45%87 24%81 88%82 74%92 13% with a different number of hidden states improved the re- Table 2 Classiﬁcation accuracy comparison sults We are convinced that a minimum system formed by a wearable badge and a cellular phone can achieve fairly good results in daily activity recognition (80%) Our results show that it is possible to to recognize some References of the most common activities using a single accelerome- ter on the chest (with 62 45% average accuracy) Adding L Bao and S S Intille Activity recognition from user- a second accelerometer on the hip or the wrist improved annotated acceleration data InProceedings of the 2nd In- our classication accuracy by approximately 20% Adding ternational Conference on Pervasive Computing, pages 1–17, a third accelerometer improved the global classication ac- Apr 2004 curacy by an additional 10% M L Blum Real time context recognition Master’s thesis, Swiss Federal Institute of Technology Zurich (ETH) & MIT The results presented so far were obtained by modeling Media Lab, Oct 2005 each activity with an HMM having two hidden states How- R W DeVaul and S Dunn Real-time motion classication ever, we think that classication results could be improved for wearable computing applications Technical report, MIT by modeling each activity with a different number of hid- Media Lab, Dec 2001 den states In some cases, two hidden states might not be P Lukowicz, J A Ward, H Junker, M Stager, G Troster, enough for capturing the different stages of a particular ac- A Atrash, and T Starner Recognizing workshop activity using body worn microphones and accelerometers InPro- tivity, especially when the activity involves different move- ceedings of the 2nd International Conference on Pervasive ments and body positions Previous studies have not taken Computing, pages 18–22, Apr 2004 this into consideration and have modeled all activities using U Maurer, A Smailagic, D P Siewiorek, and M Deisher the same number of hidden states Activity recognition and monitoring using multiple sensors Table 3 shows the K-fold cross-validation accuracy when on different body positions InInternational Workshop on Wearable and Implantable Body Sensor Networks , Apr 2006 modeling each activity with two-hidden-state HMMs and E Munguia Tapia, N Marmasse, S S Intille, and K Larson when using the number of hidden states,Qmin, that gives Mites: Wireless portable sensors for studying behavior In the minimum test error for conguration C4 of accelerome- Proceedings of the 6th International Conference on Ubiqui- ter placement Based on these results, we select the number tous Computing, Sep 2004 of hidden states,Qopt, that best models each of the activities K Murphy http://www cs ubc ca/murphyk/Software/HMM/ studied in this paper 